Welcome PCI Networking Card Vendors to the Macintosh!
For some of you this is the first time developing for the Macintosh while others are just migrating from NuBus to PCI. As the Macintosh hardware platform moves from NuBus to PCI, the software networking architecture is also changing. The new architecture, Open Transport, is based on the STREAMS environment where protocol providers communicate with each other through messages. The lowest layer (driver) in this architecture is the DLPI (Data Link Provider Interface) which is based on a standard that is maintained by UNIX International.
The sample networking driver found in this folder will demonstrate how to build a native (PowerPC) DLPI for your PCI Networking card. The DLPI will enable your card to function with Open Transport and the Macintosh OS.
This sample does not address any issues surrounding OpenFirmware. Sample code for that environment can be found in another folder on the DDK.
Before reading this document you may wish to read:
Designing PCI Cards and Drivers for Power Macintosh Computers by Apple Computer
Data Link Provider Interface Specification by UNIX International, Revision 2.0
Streams Modules and Drivers by Prentice-Hall
In order to actually build the sample you will need the following pieces:
Macintosh On RISC SDK CD - MPW environment for generating PowerPC code
or Metrowerks for generating PowerPC code
Open Transport Developer Files - header files and libraries
Driver Libraries - libraries for Native drivers, NameRegistryLib, PCILib, and DriverServicesLib
Driver Header Files - header files for the native libraries
Now that we have all the pieces collected we can start on our journey to creating a working DLPI for your PCI Networking card! If you are like me, I like to try things before I read the documentation. So let's jump right in and modify the DLPI to work with your card. First you need to go into the HWSpecific.h file and change the two strings that contain the name of the card. If the card does not contain any OpenFirmware information then just modify the strings to list the vendor ID followed by device ID. This information should match the values found in the card's PCI configuration space. After this is finished then rebuild the DLPI following the instructions in the appropriate Build Commands file.
Once the new DLPI is built, it can be dropped into the Extensions Folder and your PCI Networking card can be plugged in. After the machine boots to the Finder, go into the AppleTalk Control Panel and observe the new Network Connection that is available. The new connection represents your card on the PCI bus. The connection can be enabled by selecting the Network Connection and responding to the information dialog. The System is now sending messages to your DLPI which is responding appropriately.
Now that was pretty simple to get up and running, and people told you that the Macintosh was hard to program!
By changing the device strings in HWSpecific.h we were able to get the System to match the PCI card with the driver. Messages are actually getting sent to the DLPI, but nothing hardware related is happening since we have not adapted the DLPI for the specific card. The routines that are hardware specific can be found in HWSpecific.c. This file contains all the skeleton routines that must be completed, in order for the card to send and receive packets. The following routines need to be modified:
The DLPI currently can built using MPW's Macintosh on RISC development environment or Metrowerks PowerPC build environment. Please read the Build Commands file found in each of the build environments folder.
Validating Hardware
In some cases multiple card vendors will be using the same PCI Networking chip set, but writing their own drivers. Most chip sets have their PCI configuration space built into a chip. The system either uses the vendor id and device id, or a card developer "name" property in OpenFirmware FCODE to match a driver. The "name" property is the preferred solution because the system can be sure to correctly match a driver with its device. If no FCODE is supplied, then the driver to device matching is based on the vendor id and device id that is associated with the chips on the board rather than the board itself. This means that cards from two different vendors may look identical to the Macintosh OS. The solution to this problem is to call each driver and "ask" it about a certain hardware device. The ValidateHardware entry point to the driver should be used by the DLPI writer to validate that their driver will work with the device described by the NameRegistry ID passed into the routine. Using the NameRegistry ID the driver has enough information to start communicating with the card. Use the probe routines in the Driver Services library to access the card at ValidateHardware time. This routine will handle errors that occur when accessing hardware that is not available.
Accessing Hardware
When writing to the hardware it is important to know how your card is mapped into the Macintosh Memory Map. The mapping of the card's space is automatically handled by the boot software. It is the responsibility of the DLPI writer to enable the appropriate (I/O or Memory) space in the PCI command register found in the PCI configuration space. Do not forget about the Bus Master bit if your card has DMA! All DLPIs must use the PCILib routines to read and write PCI configuration space. These routines find your card, based on the RegEntryIDPtr, and handle the endian issues between the processor and the PCI card. When reading and writing configuration space you only need to pass in the offset in configuration space. For example to read the PCI configuration command register, the address passed into ExpMgrConfigReadWord would be 0x00000004.
If your card uses I/O space to map it's registers then the DLPI has the option to use the PCILib routines to read and write the hardware (ExpMgrIOReadWord, ...) or virtual addresses. The preferred method for both I/O and memory mapped register sets is to obtain the "AAPL, address" property value (virtual address) which is a logical address that can be used by "C" pointers. Keep in mind that when using pointers the data that is read and written needs to be endian swapped for the operation. The NameRegistry ID is all that is needed when talking to PCI config space. For cards that use memory space and I/O space the driver writer must extract the register set address of the card from a property in the NameRegistry, an example of this can be found in PCIRoutines.c. In the sample DLPI provided, the fictional card only has I/O space and not Memory space. This code is shown in HWSpecific.c in routine ABCVendorIsThisOurCard().
Device Properties
The Name Registry contains information about various items in the System. PCI cards can be found in the Name Registry along with their various properties. A couple of the properties are very important to the DLPI writer. The assigned address by the boot software can be found in the "assigned-addresses"/"AAPL, address" property. This property may be an array of AssignedAddresses structures if your card supports multiple address spaces. In order to communicate with your hardware, beyond the configuration space, the DLPI writer must read in this property. Another property that is important is the "driver-ist" property. This information is used to register the card's interrupt service routine. Examples of how to read these properties can be found in the sample DLPI.
Interrupt Service Routines
In the interrupt service routine the DLPI should defer as much work as possible to the "callback" routine. I/O subsytems that "hog" System resources tend to accomplish their task quickly, but the end result is a bad experience for the user. The cursor may not update correctly or the user may not be able to switch to different applications. Please keep this in mind when implementing the interrupt service routines.
Driver Services
DLPI writers should only use services offered by the Driver Services Library and Open Transport. Both environments offer memory allocation and a deferred task mechanism. The choice of which to use is up to the DLPI writer. Those developers that have developed for the Macintosh in the past should pay close attention to what OS services a DLPI can use. It is important to keep in mind that DLPIs that you are currently developing should work on the current Macintosh OS and the new "modern" Macintosh OS. If your DLPI plays by the rules laid out in the Apple PCI documents then you should have no problem. A couple areas to watch out for are memory allocation (use OTAllocMem or PoolAllocateResident) and locking memory down and getting its physical address (only use PrepareMemoryForIO and CheckpointIO). The DLPI should not open/read/write/close any types of files.
Driver Design
When the ABCVendorInitialize() routine is called the driver allocates the DLPI private data structure. This structure is accessible anytime in the driver since the pointer to the structure is stored in a global called gDLPIPrivateData. In the PowerPC runtime environment code fragments (which our DLPI is) can have globals. These globals are available anytime you are executing your DLPI (normal threads, interrupt service routines, deferred tasks, ...). Of course each field in the DLPI private data structure could be a separate global, but to me it is easier to put everything in one structure and then just dereference the pointer. This data structure can be found in HWSpecific.h. As you fill out your hw specific routines you can just add new fields to this data structure. I could have made the entire structure a global as opposed to just a pointer, but when Open Transport loads our DLPI it must get a contiguous chunk of memory equal to our code size + our global data size. By allocating the data structure, the Open Transport loader has a better chance of fitting our DLPI into a contiguous free space. Of course this also means that we may fail during our initialization because the DLPI could not allocate memory for the DLPI private data structure.
The DLPI private data structure contains information relating to the DLPI and any hardware it handles. Each time a new stream is opened a new DLPIStream structure is allocated. The DLPIStream structure is allocated using OTAllocMem. The first element in the DLPIStream is a link to the next stream. The DLPI private data contains a queue of all the streams opened for this port. This queue (linked list) is a convenient way to walk through the list of streams. When a stream is closed then it is dequeued from the stream queue and the memory is deallocated. The DLPIStream structure can be found in DLPIRoutines.h.
Improvements for the Next Release
• An algorithm is needed to handle flow control when sending packets
• No statistics are returned from the DLPI
• The Power Services calls have not been implemented
In the Metrowerks project the InterfaceLib was deleted because it was not needed. A new library was been added to the project because the OpenTptModuleLib was built with a entry point for the same routine twice. MPW just ignores this, but Metrowerks considers this an error so the OpenTptKernelUtilLib has been added because it also defines the routine, but only once. By adding this routine, though causes Metrowerks to complain about 93 warnings of duplicate definitions.
In DLPIRoutines.c
The BuildTxPacketHeader():
packetHeader->f8022Part.fCtrl = ctrl;
if (destAddrLen >= kEnetAndSAPAddressLength + k8022SNAPAddressLength)
should read:
packetHeader->f8022Part.fCtrl = ctrl;
if (destAddrLen >= kEnetAndSAPAddressLength + k8022SNAPLength)
A developer has pointed out a problem with 802.2 TEST and XID in the DLPI. This functionality is not properly implemented, but it will probably not be corrected for awhile. Apple is working on how this process should actually work. The main impact of this problem with be for Token Ring developers. We will keep you posted.
EntryPoints.h
Open Transport no longer requires that OTInitModule() and OTTerminateModule() be called, but I have left them in just because I get nervous about taking things out at f3 time when things are not supposed to be changed. I have included prototypes for the routines in EntryPoints.h so that CodeWarrior will not complain. The OT link library contains the stubs so everything will link properly.
Please note that SetInterruptTimer does not work properly for drivers. You can use the Time Manager functions to achieve the same functionality.
Some developers are having problems with CheckPointIO() returning an error or crashing during machine shutdown or restart. We are still investigating the cause of the problem, but one option is just to not call CheckPointIO(). On PCI based machines this routine does very little since the machines have cache coherency.
Another problems that developers are seeing is not being able allocate contiguous memory when VM is on. One approach is to make the allocation count = buffer size + (page size -1) using NewPtr and then find the page boundary in the buffer and at that point call LockMemoryContiguous(). This works on buffer sizes that are less than a page size. The DriverServicesLib contains a routine to get the page size. Another technique for allocating contiguous buffers is to call NewPtr then call LockMemoryContiguous if this fails then recall NewPtr and recalling LockMemoryContiguous. Do not deallocate the first buffer because the Memory Manager will just give it back to you during the second calling sequence. For each buffer you should set a retry count of possibly 4 or 5.
As you may have noticed all the solutions contain calls to Toolbox routines which we have always lobbied you not to use. NonToolbox calling drivers will be compatiable with Copland, but right now I see no other way of working around these problems.
1.0B5 (B4 Sample was never released) 5/12/95 with Open Transport b4c1
EntryPoints.c
In the DriverDescription we no longer need the kOTModIsPortDriver flag.
DLPIRoutines.c
In FindStreamForReceivedPacket() the result from dupmsg should be checked for NULL so that a NULL message is not passed to HandleReceivedPacket().
In DoGeneralInfo() the value returned in dl_max_sdu depends on whether the stream is in raw packet mode or not. If the stream is not in raw packet mode then the correct value should be the Ethernet packet size - the protocol header. This must be changed in order for TCP to work in the next release of OT.
All files that include any Open Transport header files must use OpenTptModule.h as the first include file. OT uses this file to setup compile time flags that are for modules as opposed to clients.
The GetPCICardBaseAddress() has been rewritten (again). This time the routine can be called multiple times to find the logical address for all of the address spaces of a card. The indicator to get each address space is the offset that corresponds in the config space. Each address space has a config base offset register. Each card can have 6 space registers and 1 ROM space as defined by the PCI Rev 2.0 Chapter 6. Please read the comments associated with the routine in PCIRoutines.c. This change is reflected in EntryPoints.c, PCIRoutines.c, and HWSpecific.c.
The link options in the MPW makefile now explicitly set the output format of the PPCLink to pef. Previously the default output was being used which was pef, but that could change sometime in the future so it is better to specify it explicitly.
A sap of 0x0000 has now been made a special sap in that when this comes down to the susbind routine it indicates that the client wishes to receive all 802.2 packets. This is a very similar feature to the old .ENET API in that attaching a protocol of 0 meant that the client wants all the 802.2 packets that come in through the driver's port. This feature has been added to the sample and only affects DLPIRoutines.c and DLPIRoutines.h. For details on the implementation checkout FindStreamForReceivedPacket() and SubsBindTheStream().
Some clients may wish to send and receive raw packets so a new ioctl must be supported by the DLPI. In order to support full size raw packets the dl_max_sdu field (in DoGeneralInfo()) must report the actual size of a packet. The IOCtlTheStream() has been updated to handle the new case in the switch statement. Any of the other changes to add this feature are reflected in DLPIRoutines.c and DLPIRoutines.h. To find all the changes just search on kReadRawPackets.
EntryPoints.c
In the IOCTlTheStream() the enum name has changed from kOTSetFramingType to I_OTSetFramingType.
Removed the Kernel compiler flag at the top of each of the .c files because Open Transport no longer needed it.
Updated the InstallISR and UninstallISR routines so that they match the PCI Cards and Drivers example in Chapter 9. The change did not affect the functionality of the routines except that the ISR property is now on the stack instead of being allocated from a memory pool.
Changed some minor things in the MPW makefile to make it more readable.
Corrected various warnings with the MrC compiler. Now the sample can be compiled with the MrC compiler by replacing "PPCC -appleext on" with mrc in the makefile. Take your pick of MPW (PPCC, MrC) or Metrowerks!
Modified the handling of out of memory condition for DoWriteScheduler(). If the bufcall routine fails then a timer message is setup to reenable the service routine after 500 ms (1/2 second). The message is allocated in the OpenStream() and deallocated in CloseStream(). There is a new routine called HandleTimerMessage in EntryPoints.c which determines if the timer is valid and then checks what kind of timer message it is, right now we only have the buffer timer message.
A new file has been added to the Metrowerks folder which allows you to define compilation flags. This file will be included during the compilation of each file. In mpw the flags would be entered on the command line.
PCIRoutines.c
Changed GetPCICardBaseAddress() so that it uses the "AAPL, address" property for the register set address. This routine now takes an additional parameter called an index which determines which value in the address property should be returned. The caller of the routine must know which order the addresses will be in. If your card uses I/O and Memory space for different register sets then pass in 0 and then 1, but you must know if the 0 index address is the I/O or Memory mapped address. The sample passes in 0 since the example card only has I/O space.
DLPIRoutines.c
Removed ClassifyRxPacket() from the file since it was not being used.
The version number 1.0A8 now correlates to the PCI DDK CD version number which matches the ROM and System Release number. This will clear up any confusion about which version of the sample DLPI should be used with which version of the ROM and System.
The DLPI can now be built using Metrowerks!!!!! Read the Build Commands file in the Metrowerks Build folder.
A new file has been added to the sources (EntryPoints.h). Each .c file now has a corresponding .h file. All the function prototypes have been moved to the appropriate .h files.
DLPIVersionNumber.h was removed from the sources. The version number was moved into the ResourceTemple.r file.
Removed all references to timeout() and untimeout(). The code now only uses the bufcall() to enable the write queue after failing to get memory.
This version of the DLPI Sample must be built with Open Transport a5. You must use the new PPCLink MPW tool which is included on this CD for the MPW build. Rename the current PPCLink tool (which resides in the Tools folder inside the MPW folder) to PPCLinkOld and then toss in the new PPCLink tool. The makefile will not build the sample correctly unless the new linker is used. The reason for the change in linkers is that OT has changed the format of the library that the sample links with.
The issue of multicast addresses needs to be discussed for a moment. It is important that your HW specific code keep a usage count of each multicast that is enabled. In the DLPIRoutines.c file the enable multicast routine will check the current stream to make sure that the new multicast is not already registered. If everything goes okay then the ABCVendorRegisterMulticast() routine is called. This routine should check if the hardware has already registered this multicast, if it has then it should just increment a count and return. The reason for this is if two different streams register for the same multicast and then one of the streams decides to deregister the multicast. If there is no usage count then the HW specific code will disable that multicast wherein reality one stream still wants that multicast to be enabled.
There is now a probe routine that can be used by your ValidateHardware routine to check if the card pointed at by the Reg ID is actually your card. Normally this routine would be used to check some special information (register, ...) that identifies the card as yours. If the card you were matched with is not your card then the reading of a special register may cause a problem (bus error, ...), so this routine should handle the problems. I have not used this routine in the Sample DLPI.
A new technique has been devised for communicating with I/O Space cards. There will be a new property in the Name Registry for your card called "AAPL, addresses". This property is used in conjunction with the "assigned addresses" property to get the "real" address for an I/O address space (and memory space). Using information from the new property it is possible to directly address your I/O space using 'C' pointers. Please read about this in the documentation enclosed on the CD. I will implement an example of this in the next DLPI Sample. The current scheme of using the Expansion Mgr and the "assigned addresses" property will continue to be supported.
DLPIRoutines.c
In the WriteServiceRoutine some routines now check if a retry is necessary because a message buffer could not be allocated.
PCIRoutines.c
In GetPCICardBaseAddress() I did not dispose of the Property memory that was allocated during the GetProperty call.
The error value that Install and Uninstall ISR routines returned was changed from OSErr to OSStatus because the Native driver APIs (and Open Transport) have changed from OSErr to OSStatus.
The "driver-ist" now contains three entries for interrupst; chip, dma output, and dma input. For PCI cards you only need to use the chip entry because all interrupts from a PCI card are OR'd together so the motherboard only views the PCI interrupts as a chip interrupt. That is why I use the 0 index value for the "driver-ist" property.
HWSpecific.c
In ABCVendorIsThisOurCard() I should use the kIOSpaceEnableBit to enable the I/O Space of the card instead of kPCIIOSpace (luckily they have the same value). If you want to enable Memory Space on a card use kMemorySpaceEnableBit (defined in PCIRoutines.h).
In ABCVendorInterruptServiceRoutine() the OTEnterInterrupt/OTLeaveInterrupt routines no longer need a paramater passed in.
dlpiCFM.make:
Removed some link files that were not necessary. The following files were commented out: InterfaceLib.xcoff, StdCLib.xcoff, and StdCRuntime.o. The linker can now do the makepef and the makesym all in one pass. The symbol file for the two machine debugger is now placed in the Built folder for the MPW build.
DLPIVersion.rsrc is in the Metrowerk's build folder. It contains a version resource that will be included when the DLPI is built using CodeWarrior. It contains the same information as the ResourceTemplate.r file found in the common sources.
Must build with Open Transport 1.0a3 or greater using the Open Transport SDK Folder. Open Transport renamed and reorganized the Developer Files Folder into Open Transport SDK. The build file has been updated to reflect the Open Transport folder changes.
The sample code discusses copying your received packets from your DMA buffers to a Streams message block. Another approach is to pass your DMA buffers and not do any copies. The Streams routine to accomplish this is called esballoc, but a more convenient Open Transport call is OTAllocMsg. I recommend using the OT routine because for one the esballoc routine is currently not working correctly and number two, the OT routine does not need a parameter block associated with the DMA buffer. So if you want to use these routines please read about them in the Streams book.
A copy of the DLPI Specification can now be found on the CD in PostScript format.
File: PCIRoutines.c
Now using definitions from the PCI.h file for Assigned Addresses structure and bit assignments.
File: DLPIRoutines.c
The FindStreamForReceivedPacket() routine was modified to correct some problems with extracting the sap values from the received packet. The sap location is dependent on the type of received packet. Also corrected a problem with accepting multicast addresses. The flag for checking the multicast had a copy/paste bug were the snap flag was used. The packet type is now passed to the HandleReceivedPacket() routine so that we do not have to classify the packet type in both routines.
File: HWSpecific.c
In ABCVendorInitialize() after the gDLPIPrivateData is allocated a check is made to see if the allocation was successful. The check is incorrect it should read if (gDLPIPrivateData == NULL) instead of if (gDLPIPrivateData).
In ABCVendorIsThisOurCard() we now correctly enable IO Space instead of Memory Space. Our example is based on a card that only has IO Space.
